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Abstract. - The dynamics based on information transfer is proposed as an underlying 
mechanism for the scale-invariant dynamic critical behavior observed in a variety of systems. 
We apply the dynamics to the globally-coupled Ising model, which is analytically tractable, 
and show that dynamic criticality is indeed attained. Such emergence of criticality is confirmed 
numerically in the two-dimensional Ising model as well as the globally coupled one and in a 
biological evolution model. Although criticality is precise only when information transfer is 
reversible, it may also be observed even in the irreversible case, during the practical time scale 
shorter than the relaxation time. 



There exist rich examples of scale invariance in nature, manifested by the abundant power- 
law behaviors of characteristic spectra, apparently without fine tuning. This is in sharp 
contrast with the scale invariance present exclusively at a phase transition point, for which 
renormalization-group techniques have been developed [1]. As a possible framework to explain 
such scale invariance or criticality without fine tuning, the concept of self-organized criticality 
(SOC) has been proposed and, in its most broad sense, used widely [2]. However, the general 
theory for ubiquitous emergence of criticality is still lacking and most existing works are studies 
of specific models on the case-by-case basis. As an attempt to seek a general theoretical answer 
to the question why criticality appears so common, we note that essentially any system in 
nature is coupled to the environmental surroundings and consider information transfer between 
the system and the environment. The mathematical formulation of information was given 
in the context of the communication theory whereas entropy is identified with a measure 
of missing information [3]. The importance of such information transfer together with the 
role of entropy has been addressed in biological evolution based on random mutation and 
natural selection [4]: In general, every species tends to minimize its entropy, or in other words, 
attempts to get information from the environment. Here we propose that this information- 
transfer dynamics may serve as a generic and universal mechanism for dynamic scale-invariant 
behaviors observed in a vareity of systems, including physical as well as biological systems. 
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Our proposal is then supported by analytical and numerical studies of the Ising models as 
well as biological evolution models. 

We consider a system consisting of N elements, the configuration of the zth of which is 
represented by ai in general, together with its environment. The total entropy St of the whole 
is given by the sum of the entropy S of the system we are interested in and the entropy So 
of the environment: St — S + So. The equilibrium probability P{(t) for the system to be in 
configuration a = {cti, a2, ■ ■ ■ , ctjv} should be proportional to the number fio {— with the 
Boltzmann constant ks = 1) of the corresponding accessible states for the environment [4]: 

P(ct) oc e'^" = e^*"^ ee Ce"^. (1) 

When the information transfer between the system and the environment takes place in a 
reversible way, the total entropy does not change in time, keeping C constant. (The case of 
irreversible information transfer will be discussed later.) 

The time evolution of the system is then governed by the master equation for the proba- 
bility P{(j; t) of the configuration a at time t: 



dP{a; t) 
Jt 



Y^[w{a' ^ a)P{a'- 1) - w{a ^ a')P{a; t)]. (2) 



The transition rate wla a') from configuration a to a' = {(7[, a'2 cf'j^} satisfies the 
detailed balance condition 



"'(^ ~^ ^ ^-{S[E{a')]^SlE{a)]} ^ 
w{fj' (?) ' 



(3) 



which ensures that P{(j) in Eq. |^ is reached in equilibrium. Note also that the entropy in 
general depends on the configuration through the energy E (or the fitness function in the 
evolution model [4]). As a simple choice satisfying the detailed balance condition Q, one 
may take the transition rate in the form [5] 

Tow{a -^a) = " 2 * ~ j ' ^ ^ 

where AS* = S[E{a')] — S[E{(j)] is the entropy change and tq sets the time scale. 

It is straightforward to obtain the solution of the above master equation for the globally- 
coupled Ising model with the Hamiltonian 

J ^ 

where cr^ = ±1 now represents the Ising spin at site i and the coupling strength J is set equal 
to unity henceforth. We need to consider the flip of a single spin, i.e, a'f, — —a^ and a[ = ai 
for i ^ k, under which the entropy change AS" in Eq. is obtained as follows: The number 
of configurations in which there are iV_|- up-spins (having the value +1) and iV_ down-spins 
(the value —1) is given hy Vl = A^!(A^+!iV_!)~^, which gives the entropy S{M) — Inil as a 
function of the total spin M = — iV_ [6]. Under the single-spin fiip, the total spin M 
changes to M' — M — 2(Tfe, and the corresponding entropy change AS", combined with Eq. Q), 
leads to the transition rate 

TV - GkM + 2 
""'^^ = 2ro(^-H) ■ 
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Multiplying both sides of Eq. Q by Ui and taking the summation over all configurations 
(7, we obtain 

dm 2 

where {<7i{t)) = aiP{a] t), being independent of i, has been recognized as the magnetiza- 
tion m per site. It is thus concluded that the relaxation time r defined by m{t) ~ exp(— i/r) 
scales with the system size as 

T - iV (8) 

for large TV. Similarly, the correlation function {ai{t)aj{0)) can be computed by multiplying 
Eq. Q by cr.icr^ with the initial configuration a' imposed at time t — 0. This leads to 
the same relaxation (M(t)M(O)} ~ exp(— i/r) with M = J^i'^i- The energy relaxation 
also obtains the same form, with half the relaxation time. Physically, r describes the time 
scale required to overcome the entropic barrier [7] separating the two states m = 1 and 
m = —1. The above scaling behavior has a profound implication that the information-transfer 
dynamics does not possess a characteristic time scale in the thermodynamic limit, which may 
be interpreted as that the system becomes critical in equilibrium. Indeed the power spectrum 
'P(w) ^ (w^ + T~'^)~^ at frequency lu, easily obtained from the exponential relaxation, reduces 
to the pure power-law form 7^(w) uj^'^ as r grows large with N; such power spectra have 
been observed in a number of specific systems studied mostly as models for SOC [8]. The linear 
dependence on the system size N also implies that the dynamic critical exponent z defined 
by T ^ with the linear size L has the value four since the upper critical dimension of the 
Ising model is well known to be four. Equation Q, yielding dm/dt = in the thermodynamic 
limit, corresponds to conserved dynamics, for which the mean- field value z = 4 is expected [9]. 

From the viewpoint of numerical simulations, it is of interest that the detailed balance 
condition in Eq. precisely corresponds to the entropic sampling algorithm [4, 10]. Namely, 
dynamic simulations based on the entropic sampling algorithm naturally describe the time 
evolution of the system in which the information transfer to and from the environment is 
reversible. In that algorithm [10], one initially starts from S{E) — for all values of the 
energy E, and obtains the histogram H{E) for several Monte Carlo sweeps (MCS), which is 
then used to estimate a new value of S{E): 

r S{E) foTH{E)^0, 
^ ' \ S{E)+ In H{E) otherwise. 

As the above procedure proceeds, S{E) approaches the true entropy up to an additive constant; 
this is independent of E and may thus be subtracted, according to the condition that the 
minimum of S{E) vanishes. Once the correct entropy is obtained, the time evolution under 
the entropic sampling should satisfy the detailed balance condition in Eq. Q and we take the 
following procedure: (i) Generate configuration a' which differs from a only at a single site, 
say, k. (ii) Compute the entropy change AS" = S[E{(7')] — S[E{(t)]. (iii) If AS" < 0, accept 
the try and change a to a'; otherwise, accept the try with the probability e^'^'^. One sweep 
of the above procedure for all N constituents in the system corresponds to one time unit. 

In the above manner we perform entropic sampling dynamic simulations on the globally- 
coupled Ising model, and compute the relaxation time r from the correlation function {M{t)M{0)). 
In practice, assuming ergodicity and time-translation symmetry in equilibrium, we take the 
time average over t' in the expression {M{t+t')M{t'))t', and display the results in Fig.Q The 
correlation function, shown in (a), indeed follows the exponential relaxation (M(t)M(O)) ~ 
exp(— t/r). The relaxation time r is obtained from the least-square-fit and plotted as a func- 
tion of the system size N in (b), which confirms the analytical result t ~ iV. The power 
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Fig. 1 - Entropic sampling dynamics in the globally-couplod Ising model, with quantities expressed 
in arbitrary units, (a) Correlation function {M{t)M{0)) in the system of size A'^ = 1600. The dashed 
line represents the exponential function exp(— f/r) with the relaxation time r = 279. (b) Relaxation 
time T versus system size N in the log-log scale, displaying the relation t ~ AT represented by the 
solid line, (c) Power spectrum 'P(cij) as a function of frequency lv, obtained from the Fourier transform 
of the correlation function in the system of size N = 1600, in comparison with the power-law form 
V{oj) ^ oJ~'^ represented by the straight line. 



spectrum V{io) is also computed as a function of the frequency ui and shown in (c). We have 
considered various sizes and observed that the frequency region described by the power-law 
^{uj) ^ Lo^'^ indeed extends with the size A^, which manifests the emerging critical behavior 
in the thermodynamic limit. We have also computed the power spectrum of the energy and 
found the same behavior, in agreement with the analytical result. 

It is necessary to check whether the scale-invariant criticality, observed above in the 
globally-coupled system, is merely an artifact of the mean-field model. For this, we study 
the two-dimensional Ising model on L x L square lattices, which contains only local interac- 
tions and thus provides a more realistic system. The price of this is lack of analytic tractability, 
and accordingly we rely on numerical methods. We thus follow the entropic sampling algo- 
rithm to compute the entropy S{E) first and also the specific heat from S{E). The latter 
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Fig. 2 ~ The two-dimensional Ising model, with quantities expressed in arbitrary units, (a) Power 
spectrum Viu)) of the magnetization in the system of size L = 32, together with the power-law form 
represented by the straight line, (b) Relaxation time r versus the system size L: Observed is 
T ~ LF"^ , represented by the solid line. 



is found to display a peak near temperature T = 2.27 in the large system of L = 64, the 
comparison of which with the precise critical temperature = 2.2692... demonstrates the 
reliability of the obtained data. Via the same procedure, we measure the magnetization and 
the energy, as well as their power spectra [see Fig. |2Ia)]. The relaxation time r, estimated 
from the relaxation data, is shown in Fig. Efb) . Observed is the algebraic increase with the 
size: r ~ with z « 2.6. The nontrivial value of the dynamic exponent z suggests that the 
observed criticality is a genuine emergent property resulting from cooperative phenomena and 
may not be explained by simple rescaling of simulation time steps. 

In order to establish the universality of these findings further, we also investigate one of 
biological evolution models, where random mutation and natural selection leads naturally to 
entropic sampling dynamics [4]. Here we focus on the dynamic behavior of the mutation rate, 
i.e., how the number of mutations during given time interval varies with time. FigureEldisplays 
the typical behavior in the one-dimensional system of 256 species with the Bak-Sneppen type 
fitness function [4,11]: (a) time series of the mutation rate R{t) compiled in time steps of 
10 MCS and (b) the corresponding power spectrum 'P{lo). (We have varied the time step 
and considered various forms of the fitness function also in higher dimensions, only to find no 
qualitative difference.) Again confirmed is the SOC behavior, characterized by the absence of 
the characteristic time scale and 'P{ui) ^ lo^°' with a = 1.5 ± 0.1. It is pleasing to note that 
mutations correspond to extinction events of species, for which fossil data indeed yield similar 
behavior of the power spectrum [12]. 

From these studies of several model systems, we expect the ubiquity of emerging dynamic 
criticality regardless of the details of the system, as long as the dynamics is directed by 
information transfer to and from the environment. In this respect, the information-transfer 
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Fig. 3 - The biological evolution model in Ref. [4], with quantities expressed in arbitrary units, (a) 
Time evolution of the mutation rate R{t) with the data compiled in time steps of At — 10 after the 
initial transient time 2.5 x lO''. (b) Power spectrum V(uj) of R{t), exhibiting characteristic power-law 
behavior P{io) ~ u)~^'^ represented by the solid line. 



dynamics as a general mechanism for emerging criticality appears to provide a theoretical 
explanation why scale invariance is observed so common in nature. 

Finally, we consider the situation in which information transfer is not reversible. In this 
irreversible case, the total entropy does not remain constant but increases as the energy (or 
the fitness) of the system is reduced (or raised) via information transfer. We expand the total 
entropy around some reference energy (or fitness) Eq'- 



St{E) « StiEo 



(E E)^^' 



(10) 



and obtain C = 
entropy 



in Eq. Q), i.e., the probability depending on the energy as well as the 



P{a) cx e 



(11) 



where /3 = —{dSt/dE)Eo controls the amount of irreversible information transfer. In the 
absence of the energy term (/3 = 0), information transfer is fully reversible and the entropic 
sampling is recovered; conversely, the absence of the entropy term leads to the standard 
Metropolis Monte Carlo algorithm. The transition rate corresponding to the equilibrium 
probability in Eq. Hll|l may be taken as 



w{cT — > a) = 



1 

2^ 



l-ta.nh-{AS + (3AE) 



(12) 



In the globally-coupled Ising model, this leads to the evolution equation for the magnetization 
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(in the thermodynamic hmit): 

dm m + tanh /? Jm 

dt 1 + TO tanh (3Jm ' 

which in the reversible hmit (/3 0) reduces to Eq. (TJ with N ^ oo. When f3J is sufficiently 
small, a simple expansion gives dm/dt = (3J{m — m?), which is easily solved to yield the 
relaxation time r = (2/3J)~^. Strictly speaking, as soon as (3 is raised from zero, the relaxation 
time becomes finite, destroying scale invariance. However, when the irreversibility is weak, 
[3 may be so small that the relaxation time can still be much longer than the observation 
time. Then deviations from the scale-invariant behavior may not be observed in the practical 
sense [13]. 

In summary, we have investigated dynamics based on information transfer, revealing the 
emergence of dynamic critical behavior. This is implicative to a wide range of phenomena in 
biology and sociology [14] as well as in physics, and expected to be of broad applicability. 
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